Repository navigation
Conversation
…caffolding scripts/lib/routes.mjs and scripts/lib/expect.mjs port the gateway's route matcher (OAuthProvider.path_allowed?/3, AccessPolicy prefix matching, Connector.safe_path?/1) and expectation evaluator (Connector.Expect) from sprites-api. v1/vectors.json is the parity contract both sides run; its cases come from sprites-api's route and expectation tests. check-connectors.mjs now also asserts the probe and every example are forwarded routes, every expect list is well-formed and path_param compiles. The template's probe path was not in its own allowed_paths; it is now. scripts/probe.mjs sends a connector's probe like the gateway does and can record a redacted fixture; scripts/new.mjs scaffolds a connector from the template. CI runs the node --test suite, which exercises the probe runner against a local http.createServer stub. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ch body timeouts - manifest.test.mjs derives the expected connector count from the directories instead of hard-coding eleven. - buildProbeRequest adds content-type and accept application/json for a JSON body unless the request already carries them, as Req's encode_body step does; fetch would otherwise send text/plain and */*. - runProbe reads the response body inside the transport-error handler, so a server that sends headers and then stalls returns provider_unreachable instead of throwing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Collaborator
Author
|
Addressed all three findings in the latest commit: the connector count is derived from the directories, JSON probes now carry Req's default content-type and accept headers (explicit test_request_headers still win), and body reads happen inside the transport-error handler so a stalled body reports provider_unreachable. Tests added for the last two. |
README.md says what a connector is and gives the three commands. CONTRIBUTING.md walks from `new.mjs` to the pull request, with connector.json explained field by field in the order a contributor fills it, the expect checks as a table, and the checks and probe commands at the step where they are used. Internal material (how sprites-api compiles manifests, the parity vectors, hooks) is dropped or reduced to one line. The template README now says what to put in it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Documentation points at https://fly.io/sprites; sprites.dev is only for things that connect to that domain. The sprites.dev URLs inside the Exa, Firecrawl and Linkup example request bodies are request data, not docs, and stay as exported. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
sprites-api vendors every directory under connectors/ at a tag and fails the build on a name that collides with a built-in provider, which all eleven seeds do. They now live in examples/ as worked examples and fixtures. connectors/README.md says what belongs there. check-connectors.mjs checks both roots by default and finds a named connector in either; probe.mjs resolves connectors/<name> then examples/<name>; new.mjs always creates under connectors/. CI's ajv step validates examples/ and the template, and connectors/ only when it has a manifest, since ajv fails on a glob with no matches. The editor mapping, README and CONTRIBUTING describe the two roots. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
connectors/README.md is three sentences: community connectors, one per subdirectory, shipped with a tagged release, examples under examples/. README, CONTRIBUTING, the check script's comment and the probe runner's message no longer mention how connectors are vendored or built, or that the examples came from anywhere; the examples are complete, working connectors to copy from. The limitations list says a merged connector reaches Sprites with the next release, and that a connector is data only: a provider whose responses the assertions cannot describe gets an issue. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…at the root A release imports only connector.json, icon.svg, mark.svg and README.md from a connector directory, so a nested fixtures/ directory would break it. probe.mjs --write-fixture now writes fixtures/<name>/probe_<status>.json at the repository root. check-connectors.mjs refuses any other file or directory inside a connector, pointing at fixtures/<name>/, and validates fixtures/: directories named after existing connectors or examples holding only probe_<status>.json files that parse as JSON. The checks moved into scripts/lib/check.mjs so the tests run them on sandbox layouts. README and CONTRIBUTING show fixtures/ at the root and say recordings are optional and not part of a connector. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dependency-free Node 22 scripts for contributors, with tests, wired into CI and the docs.
What is here
scripts/lib/routes.mjs: port ofOAuthProvider.path_allowed?/3,AccessPolicy.path_matches_any?/2andConnector.safe_path?/1(exact, prefix*,:namesegments with reserved literals, trailing*after parameters,path_paramwith the\A/\zanchors translated for JS).scripts/lib/expect.mjs: port ofConnector.Expectplus the manifest's[path, check, ...args]decoding and validation.v1/vectors.jsonis the parity contract. Its route, expectation and malformed-expectation cases come from sprites-api'soauth_provider_routes_test.exsandexpect_test.exs, plus cases for the rules those tests do not spell out (prefix patterns, null method, deep fan-out, per-element absence). The JS tests run every vector; sprites-api should run the same file (not changed in this PR).scripts/check-connectors.mjsalso asserts: the probe path (query stripped) is allowed for its method, every example path is allowed for its method, everyexpectin the probe and itserrorsrules is well-formed, andpath_paramcompiles. Existing checks kept. Shared logic lives inscripts/lib/manifest.mjsso the tests can run it on the template too.scripts/probe.mjs <name> --key-env VAR [--write-fixture]: builds the request as the gateway does (bearer or header auth with optional prefix,test_request_headersor the content-type default for JSON bodies,user-agent: Sprites-API-Gateway, no redirects, no retries,receive_timeoutdefault 10 s), prints each assertion, classifies non-200 responses with theerrorsrules then the 401 default, and writesconnectors/<name>/fixtures/probe_<status>.jsonafter redaction. The key is never printed.scripts/new.mjs <name>: copies the template, setsnameanddisplay_name, refuses an existing directory or an invalid name.node --test "scripts/test/*.test.mjs"and validates the template with ajv as well. CONTRIBUTING.md's Checks section and README.md's layout block are updated.GET /v1/mewas not in its ownallowed_paths; the new check caught it and the template now lists it.Layout
Two roots, same per-connector layout:
connectors/holds community connectors only. sprites-api vendors every directory here at a tag of this repository and fails its build on a name that collides with a built-in provider, so the eleven seeds cannot live here. It currently holds only a README saying so.examples/holds the eleven first-party connectors (exports of production) as worked examples and as fixtures for the tests. One moves toconnectors/only deliberately, together with the removal of its Elixir module in sprites-api.check-connectors.mjschecks both roots by default and finds a named connector in either;probe.mjsresolvesconnectors/<name>thenexamples/<name>;new.mjsalways creates underconnectors/. The ajv step validatesexamples/and the template, andconnectors/only when it has a manifest, because ajv exits non-zero on a glob with no matches.Probe recordings live at the repository root in
fixtures/<name>/probe_<status>.json, never inside a connector directory: a release imports only the four connector files, socheck-connectors.mjsrefuses anything else inside a connector and validatesfixtures/(directories named after existing connectors, onlyprobe_<status>.jsonfiles, valid JSON).Semantics I was unsure about
safe_path?only guards:nameroutes. In sprites-api an exact or prefix*pattern goes throughAccessPolicyand is not safe-path checked, so/v1/*forwards/v1/../x. The port and a vector reproduce that. If the gateway should refuse it, change sprites-api first and flip the vector.allowed_pathsis a convention, not a gateway rule. The gateway sends the probe directly, so a probe outsideallowed_pathsworks in production. All eleven connectors list theirs, and the check requires it as the task asked.integerand float-integer ambiguity. Elixir decodes1.0as a float, JSON.parse cannot tell it from1, so["n", "integer"]on1.0passes here and fails in the gateway. Likewiseinuses strict equality in Elixir (1 in [1.0]is false) but==elsewhere. The vectors avoid integral floats and say so in the header..*suffix.AccessPolicytreats a pattern ending in.*as a prefix with backslashes stripped. Ported and covered by a vector; the schema allows the spelling, though no connector uses it.expectwith hooks. A manifest whose probe relies on a hook'sprobe_ok?/1cannot be judged here; the runner reports that and exits non-zero rather than passing a bare 200.node --test scripts/test/does not recurse into a directory on Node 22, so CI uses the glob form.🤖 Generated with Claude Code